Design and Development Research: A Model Validation Case

This is a report of one case of a design and development research study that aimed to validate an overlay instructional design model incorporating the theory of multiple intelligences into instructional systems design. After design and expert review model validation, The Multiple Intelligence (MI) Design Model, used with an Instructional Systems Design (ISD) Model, was tested for use by four practicing instructional designers. Instruction developed for learners using this model was then evaluated measuring post-test and attitudinal scores with 102 participants. This report also provides a reflection on the lessons learned in conducting design and development research on model validation. The procedures and findings have implications for the processes involved in instructional design model validation through designer use and program implementation. Design and Development Research: A Model Validation Case Design and development research is “the systematic study of design, development and evaluation processes with the aim of establishing an empirical basis for the creation of instructional and non-instructional products and tools and new or enhanced models that govern their development” (Richey & Klein, 2007, p.xv). It is a practical form of research that attempts to test theory and validate practice. Numerous models exist in the field of instructional design that assist designers working in a variety of settings (Gustafson and Branch, 2002). Historically many of these models have not undergone rigorous or systematic review. In recent years, there has been an increased focus on systematically studying the processes involved in the construction, validation, and implementation of instructional design models (Seels, 1994; Richey, 1998; Richey, Klein & Nelson, 2004). Current literature focused on the implementation and systematic validations of instructional design models describe five different approaches to model validation. These include expert review, usability documentation, component investigation, field evaluation, and controlled testing (Richey, 2005; Richey & Klein 2007). This article reports one case of a design and development study implementing some of these approaches to validate The Multiple Intelligences (MI) Design Model, an overlay model used in conjunction with an instructional systems design model. The study specifically measured model use, learner achievement, and program implementation and is an example of design and development research in practice. Instructional Design, Multiple Intelligences, and the Instructional Designer In an effort to better understand this design and development model validation case, we begin with a description of the context related to the development of the model. This includes definitions and background content on instructional design, multiple intelligences, and the instructional designer. Instructional design (ID) is defined as an arrangement of resources and procedures used to promote learning (Gagné, Wager, Golas, & Keller, 2005). ID models are visual representations of the ID process and are used to guide design in many settings and for many purposes (Seels & Glasgow, 1998). They are typically a result of the combination of abstract principles of General Systems Theory and analyses of practitioner experience (Banathy & Jenlink, 2004). Instructional design is the process used to create the type and delivery of instruction. Some see design as problem solving; others view it as a process of reflection-inaction, where designers take on the task of turning indeterminate situations into determinate ones (Tripp, 1991). Learning is the acquisition of the knowledge of a skill, art, or trade, by study, and/or experiences (Lindvall, 1995). It is generally agreed that learners develop understanding for themselves in ways that differ, sometimes quite sharply, from other learners (Winn, 2004). Multiple intelligences can be thought of as a learners’ tool that facilitate knowledge acquisition. Gardner (1983) states “I have posited that all human beings are capable of at least seven different ways of knowing the world (p. xi)”. Rooted in cognitive brain research, Gardner proposes that people learn in a variety of ways and have diverse strengths and abilities which, if recognized, can be developed to enable learners to reach their potential. He defines intelligence as "the capacity to solve problems or to fashion products that are valued in one or more cultural settings” (Gardner & Hatch, 1989, p. 4). Gardner’s seven ways of knowing the world or intelligences are: verbal-linguistic, logical-mathematical, musical-rhythmic, visual-spatial, bodily-kinesthetic; and two forms of personal intelligences, one directed toward other persons, interpersonal intelligence, and one directed toward oneself, intra-personal intelligence. While learners bring their own needs, experiences, and tools to a learning situation, instructional designers must incorporate those needs and experiences into instructional strategies helping learners take ownership and responsibility for their own learning (Grabinger, 1996). The significance of knowledge construction for Instructional Technology lies in the building of environments that make it easy for learners to formulate their own understandings of the world (Winn 2004). Instructional design is a process used to create instructional products, programs, and delivery systems. Multiple intelligences theory is built on the premise that learners acquire knowledge based on their learning potential and that people learn in at least seven different ways. The first two phases of this design and development study previously reported (Tracey and Richey, 2007) included the design and internal validation of the MI Design Model. Based on research on instructional design and multiple intelligences, the MI Design Model was designed to be used as an overlay with an ID model. In order to achieve this, the theoretical foundations of multiple intelligences and instructional design were examined to guide the development of such a model. The model components were then determined and an initial model was constructed. The model was then reviewed and validated by experts in the field of instructional design through a three-round Delphi study. The result was a revised and internally validated MI Design Model (Tracey and Richey, 2007). The newly developed model had to now face a usability test by instructional designers and course validation. This article reports on the final two phases of the design and development research study, the process and results of model use by practicing instructional designers and the delivery of instruction developed for learners specifically measuring performance on post-test and instructional appeal. In addition, reflections on lessons learned when conducting design and development research on model validation are presented. Approaches to Model Validation Once designed and validated by subject matter experts (Tracey and Richey, 2007), the MI Design model was ready to be tested using two separate model validation approaches — a designer usability test while instructional designers designed and developed a course and a product impact test measuring learners after they participated in the course. These validation efforts were guided by the following research questions: Designer Usability Study 1. What are the advantages and disadvantages of using the overlay design model incorporating multiple intelligences? Factors to consider included the following: a) Ease of use; b) Ability to overlay the model with an existing design model; c) Capability to meet course intent; and d) The ease of instructional design strategy creation utilizing each multiple intelligence. 2. What is required to facilitate the effective use of the validated design model incorporating multiple intelligences? Factors to consider included the following: a) Guidance required for designer model use; b) Ability to incorporate content accuracy; and d) Ease of use. Product Impact Study 3. To what extent does the design model incorporating multiple intelligences influence post-test performance? 4. To what extent does the design model incorporating multiple intelligences influence learner confidence and attitudes? Factors considered included: a) Learner attitude toward the instructional strategies; b) Learner confidence in ability to apply content; and c) Learner attitude toward training. Designer Usability Study Usability documentation is defined as “information on the extent to which a product, tool, or model can be effectively, efficiently, and satisfactorily used in the context for which it was intended” (Richey and Klein, 2007 p.160). Designer usability documentation is one of three methods of internal design model validation (Richey, 2005) which in this case studied the ease with which the model was used by instructional designers. Similar to other design and development studies, (Forsyth, 1998), this phase included documenting the instructional designer reactions to using the MI Design Model while designing and developing a team-building course. This data, combined with the program tryout and evaluation data provided the basis for final model revisions. This internal model validation step was the third phase of the four phase study; one part of the larger design and development research project. It is important to note that this phase was limited as it included only two designers using the model. Additional validation studies with designers using the MI Design Model are necessary in order to reach conclusions regarding the model. The following section describes the designer usability participants, procedures and results. Participants The designer usability phase of the study consisted of a total of four volunteer instructional designers with Master’s Degrees in Instructional Technology. Each designer, chosen from a list of volunteers, was a current practitioner with two coming from business and two coming from education. Each volunteer was a recent graduate of the same university Instructional Technology program. The designers had knowledge and experience in instructional design with less than two years practice, but little or no prior experience with multiple intelligence theory and practice. Once the four subjects were chosen, they were divided into two teams based on an equal distribution of skills, knowledge and backgrounds. Each team, labeled Team 1 and Team 2 included one volunteer from business and one from education. Since the goal was to compare the instructional products developed between the two teams, Team 1, using the Dick and Carey Model (Dick, Carey, & Carey, 2001) and the overlay MI Design Model was the experimental group while Team 2, using only the Dick and Carey Model (Dick, Carey, & Carey, 2001) alone, was the control group. The same ID Model was used for both as the study focused on the overlay MI Design Model. Procedures The two teams of designers each received: (1) materials regarding the organization in which the newly developed instruction will be used; (2) written content on team-building, the content of the instruction to be developed; (3) audience, environment, and gap analysis information; and (4) the Dick and Carey Instructional Systems Design (ISD) model (Dick, Carey, & Carey, 2001). In addition, Team 1 received the validated overlay MI Design Model (see Figure 1). Both teams were instructed to design a two-hour instructor-led classroom based course on team-building for JARC (Jewish Association for Residential Care), a non-profit organization. ____________________________ Insert Figure1 about here ____________________________ To ensure consistency between the two resulting courses in terms of goals, objectives and content, all four instructional designers worked together to assess the needs and identify the goals of the team-building training and review of the content. The teams then separated to conduct their own learner and environmental analyses based on the information given and the model(s) they were assigned to use. Both teams worked individually to create their performance objectives based on the results of their analysis and the steps of the design model they were utilizing. The teams reconvened to review their objectives to ensure each met the same team-building goal. Based on the objectives, post-test questions to be used in the product impact study were then developed together by the two teams of designers. Upon consensus from all of the designers and a JARC expert review of the objectives, the teams then separated and worked simultaneously in the same location in two different offices during the remainder of the design and development process. Each team was provided computers, an office; any desired training materials, and a desktop publisher to create the instructor guide and participant guide templates and all finished products. Two formative evaluations in the form of an expert review and small group evaluation of a rough draft of the instruction were conducted (Tessmer, 1995). The expert review of content, objectives and learning strategies of both courses was conducted with the Assistant Executive Director of JARC, and revisions were made based on resulting feedback. A small group evaluation of the training materials from both courses and post-test was conducted to improve the materials and field test the post assessment. This evaluation was conducted at the JARC offices in the training rooms to provide similar circumstances under which the learners would interact with the materials. One designer from each team conducted the training with the materials that team produced, while the other documented learner reactions. The designers documented how to improve the effectiveness, appeal and efficiency of the instruction. Revisions of the materials and post-test instrument were made. After revisions were completed for the team-building training courses, Team 1, the team using the overlay MI Design Model, was given a Likert-Scale questionnaire using (a) strongly agree, (b) agree, (c) disagree, and (d) strongly disagree as the scale to respond to each question regarding the use of the model. Space was provided for written comments from the designers for additional feedback after each question. Because the questionnaire specifically addressed questions regarding the overlay MI Design Model, Team 2 did not receive it. Results Instruction Produced. Both teams, Team 1, using the Dick and Carey and the overlay MI Design Model and Team 2 using only the Dick and Carey Model, created an Instructor and Participant Guide for the two-hour team-building course. The courses would be held in a training room with conference tables and all requested materials. The subject matter, descriptions of the Instructor and Participant Guides and the instruction designed are in Table 1. ____________________________ Insert Table 1 about here ____________________________ Designers respond. Upon completion of the course design utilizing the overlay MI Design Model, each of the two members in Team 1 filled out a 14-question Agree-Disagree Scale evaluation and added additional comments as desired. The following is a summary of the written designer feedback from the questions from the questionnaire with the complete responses in Table 2. ____________________________ Insert Table 2 about here ____________________________ Overall, the two designers who used the MI Design Model with the Dick and Carey Model responded favorably to each component. These designers found two steps the most useful: 1) the development of assessment instruments considering the multiple intelligences of the learners; and 2) the development of instructional strategies. Considering the participants, in addition to the content, while developing the assessment instruments was new for these designers. Additionally, they found the instructional strategy step helpful when they were brainstorming to identify different activities. The designers found the model, on the whole, clear and easy to use. They indicated that it was a model that could be used in conjunction with an ISD Model. The feedback from the designers was documented for future model revisions. Product Impact Study Product impact studies address the effects of using the model through the creation of the instructional products themselves, and the impact of these products on learners (Richey, 2005). In this case, the goal of the product impact study was to externally validate the MI Design Model. Specifically, in this phase of the model validation study, the outcomes of the products created were tested in a controlled testing environment. An effort to compare the outcomes of the developed products using one model as opposed to outcomes of the products developed without the model was made. Two team-building courses were developed, one with the Dick and Carey Model and one with the overlay MI Design Model used in conjunction with the Dick and Carey Model. They were then delivered and evaluated to determine the extent to which the use of the overlay MI Design Model influenced performance on post-test and attitudes toward learning in the course. Participants The participants in the training were 102 employees of a non-profit organization JARC. The executive director of JARC volunteered her employees as they had not had any previous training on team-building and was willing to provide their offices and employees for companywide training. The employees were divided into 10 different classes, with 8 to 15 JARC employees in each group. Procedures Employees of JARC volunteered to attend a team-building course offered in one of ten sessions. Five sessions were allocated for the course designed with only the Dick and Carey model and five sessions consisted of the course designed with both the Dick and Carey and the overlay MI Design Model. The courses ran simultaneously, two per time slot. To ensure multiple intelligence representation in each course, participants agreed to complete a Multiple Intelligence Assessment (Armstrong, 1999). This was done in an attempt to include representation of all seven multiple intelligences in both courses. When arriving at the training site participants were randomly stratified according to their results on the MI Intelligence assessment into the sessions. Since the focus of the study was specifically on the overlay MI Design Model there was no need to further pre-assess the participants. One designer from each of the two designer teams instructed each of the five courses developed with their assigned model. Each designer, chosen to facilitate the trainings currently worked in education and had experience in the classroom, similar facilitation skills and styles. This ensured consistency and thoroughness in each course. The instructors set up the training rooms and were prepared with Participant Guides and all other training materials needed. Upon completion of the two-hour training, the instructors administered the post-assessment (post-test) designed to determine the levels of knowledge about team-building after training. In addition, participants were asked to complete the thirteen-item opinion survey in an attempt to measure their attitudes toward the training. The post-test, and opinion questionnaires, were then collected and labeled by course number, date and time. Individual post-tests were also labeled to match the MI Assessment participants completed prior to the training. Instrumentation Post-test assessment. A ten-item post-test was designed and evaluated to measure participants’ knowledge of team-building concepts immediately following the training sessions. The instructional designers developed the multiple choice and true/false question items during course development. Each question was reviewed by the four designers, a JARC representative, a small group of learners, and the researcher. Opinion questionnaire. Participants also completed a 13 item-opinion questionnaire to provide general reactions to the course and the materials. Responses to each of these measures used a Strongly Agree-Strongly Disagree Scale, with (1) being Strongly Agree. In addition, the questionnaire addressed perceptions of learner’s confidence levels during and after the training and general involvement in the team-building training. Results Table 3 presents the results of the differences between MI and ISD groups on overall post-test scores while Table 4 presents the differences between MI and ISD groups on attitudes toward the training. The following are a summary of the results. Performance on post-test results. The difference between the MI group and the ISD group appeared to be significant on the overall post-test score (t = 2.79, p<.01). Out of a possible ten points, the MI group scored significantly higher (mean = 8.78) than the ISD group (mean = 7.90) on the knowledge portion of the measure. These results were reviewed with caution however, as the learning performance was based on a 10 item multiple-choice and true-false test with the results indicating the MI Model being superior from an average difference of less than one question in the two groups. The outcomes could reflect a ceiling effect in the post-test scores and for model validation, these results alone provide little in the way of persuasive evidence that one model was better than the other. Therefore, in this study, the post-test results were reviewed as only one indicator validating the model. ____________________________ Insert Table 3 about here ____________________________ Attitude results. A t-test was conducted on each item in an attempt to assess any differences in opinions between the MI and ISD groups. Given the scale used, a negative t-value indicates that the MI group responded more favorably to the item than the ISD group (Table 4). ____________________________ Insert Table 4 about here ____________________________ Outcome predictors. A regression analysis was conducted in an attempt to identify those variables that were predictors of participant performance on the post-test, attitude toward the training, and confidence. Of the five independent variables used, group membership was the only significant predictor of post-test performance F (1, 99) = 6.656, p < .05 (R = .06) suggesting that the overlay MI Design Model used may have influenced post-test scores. As stated earlier however, these results were reviewed with caution. In examining the predictors of participant attitude toward the training, the items related to level of engagement, and pre-course motivation appeared to be significant predictors. A more targeted analysis of attitude, examining items related to confidence, indicated confidence as a team member and confidence in the ability to contribute to a team appeared to be significant attitude predictors. In the analysis of confidence as a team member, items related to the use of group activities in the instructional strategy, enjoyment, and group membership appeared to be significant predictors of team member confidence. Initial analyses indicated confidence in the ability to contribute to a team due to the training, feeling at ease, being involved in all activities, group membership, and the spatial and intrapersonal intelligences were significant predictors. Upon further review it was determined that both groups reported they felt confident they could operate as a team member and had something to contribute to a team (Table 4, Items 12 and 13). These results indicate that participants had similar levels of confidence regarding whether they could actually implement their learning. Table 5 provides a summary of the four analyses conducted. ____________________________ Insert Table 5 about here ____________________________ Discussion and Implications The purpose of this article was to report one case of a design and development study implementing approaches that attempted to validate The Multiple Intelligences (MI) Design Model, an overlay model used in conjunction with an instructional systems design model. The study specifically measured model use, learner achievement, and program implementation and is an example of design and development research in practice. The procedures and findings have implications for the processes involved in instructional design model validation through designer use and program implementation. The following discussion focuses briefly on the results of this study and provides a reflection on the lessons learned in conducting design and development research on model validation. Validating the MI Design Model in Instructional Design Designer use. Designer use validation of the MI Design Model focused on designer reaction to the ease in using the model, the ability to overlay the model with an existing ID model, the ease of instructional design strategy creation utilizing multiple intelligences, and the guidance designers need in order to use the MI Model while designing instruction. The designers in this study evaluated the overlay MI Design Model as being easy to work with and helpful in the design process especially during the selection of the instructional strategies. McAleese (1988) indicates that in a sense designers are engaged in the learning as well as design because the designer’s personal knowledge structures are altered by the information present in the design environment. The designers in this study commented on how the list of strategies was helpful in their creative thought process and how they customized the training to match their audience. In this sense, the designers appeared to be engaged in the learning process as well as the design process as McAleese suggests. Product impact. Product impact validation of the MI Design Model focused on how the instruction created with the overlay model influenced learner confidence, attitude, and post-test scores. Learners’ attitudes toward the instructional strategies, their confidence in their ability to apply the content, their overall attitude toward the training and their performance on the post-test were analyzed. Figure 2 illustrates the summary of the regression analyses with each box representing a variable testing the impact of the MI Design Model on learner attitudes, confidence and performance on the post-test. ___________________________ Insert Figure 2 about here ___________________________ The learning process appeared to be stimulated by the use of the MI Design Model, especially in terms of learner attitude toward the instructional strategies used. These strategies in turn appeared to positively affect learner attitudes (see Figure 2). One of the four attributes Keller (1987) believes should be considered when designing instruction that motivates the learner is confidence. For learners to be highly motivated, they must be confident that they can master the objectives for the instruction. If they lack confidence then they will be less motivated. Confidence refers to the learners perceived likelihood of success (expectancy) and whether the learner perceives success as being under his or her control (Mory, 2004). There is a body of research that shows correlations between attitude and knowledge acquisition resulting in positive post-test scores. For example, knowledge acquisition motivation, as described by J.W. Atkinson (1974) explains a person’s desire to perform and achieve. This theory sprung from behaviorist tradition that connects so-called “motivated behavior” with performance based on what he calls “resultant achievement motivation” (Williams, 1996, p.970). Learners achieve their optimal level of performance when they have an intermediate level of motivation to achieve success and to avoid failure. Learner’s positive attitudes toward the instructional strategies may have led to confidence which in turn may have led to positive post-test scores. Interestingly though, both groups reported they felt confident they could operate as a team member and had something to contribute to a team (Table 4, Items 12 and 13). These results indicate that participants had similar levels of confidence regarding whether they could actually implement their learning. The instruction took place in a work environment which may have caused results to appear more trustworthy among the designers and researchers however the richness of the work environment may have made it more difficult to prevent unrelated factors from impacting the results. Is the confidence data a result of the facilitators, the participant’s belief systems, and/or the instructional designers’ products in general? These are some of the questions surfacing in the analysis of this study’s results and initiate the need to replicate this study and to reflect on the use of design and development research and model validation techniques. Model Validation Techniques in Instructional Design ID model validation has been viewed as either internal or external. Internal validation is a confirmation of the components and processes of an ID model illustrated in this designer usability study, while external validation is a validation of the impact of the products of model use, illustrated in this product use study. As is often the case in design and development research, the internal and external model validation processes in this study were part of a larger design and development research project (Richey & Klein, 2007). The overlay MI Design Model was initially created and validated in an expert review (Tracey & Richey, 2007). Following that expert review validation study, designers used the MI Design Model. By comparing the use of the overlay MI Design Model and the Dick and Carey (2001) model used by one design team, to the use of the Dick and Carey Model alone, used by the second team, both teams were instructed to design a team-building course for a non-profit organization. The MI Design Model was then externally validated in a controlled testing study testing the effects of the products created with The MI Design Model and the Dick and Carey Model as compared to the use of only the Dick and Carey Model. The resulting programs were implemented and evaluated through the use of learner knowledge post-tests, and surveys addressing participant reaction to the instruction. Design and development processes are studied sometime during, but oftentimes after, the development process of several interventions. Its purpose is the creation of design knowledge in the form of a new or enhanced design or development model or design principles (VisscherVoerman & Gustafson, 2004). The four phases in this design and development study are insufficient to make recommendations regarding the future use of the MI Design Model; rather they are compelling enough to invite additional exploration of the model components, use, and the products developed. Design and Development Research for Model Validation Design and Development research can focus on the development, validation, or use of a model (Richey & Klein, 2007). The primary goal of this research is production of new knowledge, as in this case in the form of a new design model. Since most ID models are based on an assumption that they are valid (Richey, 2005), and the data supporting validity is often rare or nonexistent, this study attempted to collect and analyze empirical data to validate the MI Design Model. The internal validation in this study attempted to test the components and processes of the model while the external validation attempted to validate the impact of the products of model use. There are lessons learned about conducting design and development research and model validation illustrated below. Validation through designer use. A usability test ought to determine under what conditions the design model can be used. Numerous variables may come into play during a usability test so the number of designers may prove critical. In this study, the small number of participating designers makes recommendations regarding future use of the MI Design Model problematic. In addition to evaluating the model components and model use with a larger designer population, a systematic documentation of designer procedures, time taken, resources used, problems encountered and resolutions of these problems must be conducted. Reconstructing the actual practices of designers is an essential step toward developing insight into the gap in design models and design practice, and identifying their reasons for this gap (Visscher-Voerman, 2004). Designer use studies indicate the focus should be on designers’ rationales and reasons for decisions made rather than on the activities performed. Studies illustrating the difference between novice and expert instructional designers confirm that each perform design tasks quite differently (LeMaistre 1998; Rowland, 1992). Therefore, replication studies using the MI Design Model must include data gathering while both novice and expert designers create instruction on an array of subject matter in diverse settings. Lessons learned in product impact. External model validations can be complex due to the large number of extraneous factors; some which surfaced in this study. These include the characteristics of the facilitators, and possibly the learner’s past history and value system as indicated in the confidence results (Table 4, Items 12 and 13). Nonetheless, external validations address those factors that many consider to be the central focus of design efforts (Richey, 2005). Verification of model effectiveness in this study was based on measures of learning and participant reactions to the instruction. Controlled testing validation research provides data that supports the validity of a given procedural model under controlled conditions (Richey, 2005). This controlled validation study established an experiment that isolated the effects of the MI Design Model used with The Dick and Carey Model as compared to the use of The Dick and Carey Model alone. Ideally, a comprehensive validation, examining the MI Design Model used under a variety of conditions is needed. To meet this goal, future research requires systematic replication to determine the impact of factors such as, alternative settings, alternative types of learners, alternative content areas, and a variety of delivery strategies. There is a need for numerous empirical studies that clarify the processes involved in the validation of ID models and in the products created through model use. Researchers identifying indicators of the success of an ID model should question if the model is universally applicable, its ability to use it with other models, and the match between the application context and the context for which the model was originally intended (Edmonds, Branch, and Mukherjee, 1994). Replications of model effectiveness under a variety of conditions and integration of the various findings will increase the credibility of that particular model, but will also provide data to support (or refute) the fields’ assumptions that ISD processes are generic (Richey, 2005). The presence of this body of research could explain some of the processes involved in ID model use and the study of the products created in the ID process. This design and development study was an attempt to internally and externally validate an instructional design model in a systematic process rather than supporting a model based on personal advocacy. Thus, the approaches used in this study can be generalized to other empirical research involving ID modelconstruction and validation. ReferencesArmstrong, T. (1999). 7 kinds of smart: Identifying and developing your multiple intelligences.New York, New York: New American Library.Atkinson, J.W. (1974). The mainsprings of achievement-oriented activity. In J.W. Atkinson &J.O. Raynor, (Eds.). Motivation and achievement, pp.193-218. Washington DC: Winston.Dick, W., Carey, L. & Carey, J.O. (2001). The systematic design of instruction (5 ed.). NewYork: Addison-Wesley Educational Publishers.Edmonds, G.S., Branch, R.C., & Mukherjee, P. (1994) A conceptual framework for comparingID models. Educational Technology Research and Development, 42(4), 55-72.Forsyth, J.E. (1998). The construction and validation of a model for the design of community-based train-the-trainer instruction. (Doctoral dissertations, Wayne State University, (1997).Dissertation Abstracts International –A, 58(11), 4242.Gagné, R.M., Wager, W.W., Goals, K.C. & Keller, J.M. (2005). Principles of instructionaldesign (5 ed.). CA: Wadsworth/Thomson Learning, Publishers.Gardner, H. (1983). Frames of mind. The theory of multiple intelligences. New York: BasicBooks.Gardner, H. & Hatch, T. (1989). Multiple intelligences go to school: Educationalimplications of the theory of MI. Educational Researcher, 18(8), 4-9.Grabinger, R.S. (1996). Rich environments for active learning. In D.H. Jonassen, (Ed.).Handbook of research for educational communications and technology. (pp.665-692). NewYork: Simon & Schuster Macmillan.Gustafson, K.L. & Branch, R. M. (2002). Survey of instructional development models.(4 ed.).ERIC Clearinghouse on Information and Technology, Syracuse, New York.Keller, J.M., (1987). Development and use of the ARCS model of motivational design. Journal of Instructional Development, 10(3), 2-10. LeMaistre, C. (1998). What is an expert instructional designer? Evidence of expert performanceduring formative evaluation. Educational Technology Research and Development, 46(3), 21-36.Lindvall, R. (1995). Addressing multiple intelligences and learning styles: Creating activelearners. Doctoral dissertation, Saint Xavier University, Illinois.McAleese, R. (1988). Design and authoring: a model of cognitive processes. In H. Mathias, N.Rushby & R. Budgett (Eds.). Designing new systems for learning. 118-26. New York:Nichols.Mory, E.H. (2004). Feedback research revisited. In D. H. Jonassen (Ed.). Handbook of researchfor educational communications and technology (2 ed.). (pp. 745-783). New York: Simon& Schuster Macmillan.Richey, R.C. (1998). The pursuit of useable knowledge in instructional technology.Technology Research and Development, 46(4), 7-22.Richey, R.C. (2005). Validating instructional design and development models. In J.M. Spector &D.A. Wiley (Eds.). Innovations in Instructional Technology: Essays in Honor of M. DavidMerrill (pp. 171-185). Mahwah, New Jersey: Lawrence Erlbaum Associates, Publishers.Richey, R.C. & Klein, J.D. & Nelson W. A. (2004). Developmental research: Studies ofinstructional design and development. In D. H. Jonassen (Ed.), Handbook of researchfor educational communications and technology (2 ed.) (pp.1099 -1130). NewYork: Simon & Schuster Macmillan.Richey R. & Klein, J. (2007). Design and development research: Methods, strategies and issues.Mahwah, New Jersey: Lawrence Erlbaum Associates, Publishers. Rowland, G. (1992) What do instructional designers actually do? An initial investigation ofexpert practice. Performance Improvement Quarterly, 5(2) 65-86.Seels, B. (1994). An advisor’s view: Lessons learned from developmental researchdissertations. Paper presented at the 1994 Annual Meeting of the Association forEducational Communications and Technology.Seels, B. & Glascow, Z. (1998). Making instructional design decisions (2 ed.). New Jersey:Prentice Hall.Tessmer, M. (1995). Planning and conducting formative evaluations. London: Kogan Page.Tracey, M.W., & Richey, R.C. (2007). ID model construction and validation: A multipleintelligences case. Educational Technology Research and Development, 55(4), 369-390.Tripp, Steven D. (1991). Two theories of design and instructional design. In: Proceedings ofselected research presentations at the annual convention of the association for educationalcommunications and technology.Visscher-Voerman, I. & Gustafson, G. (2004) Paradigms in the theory and practice of educationand training design. Educational Technology Research and Development, 52(2), 69-89.Williams, M.D. (1996). Learner-control and instructional technologies. In D.H. Jonassen,Handbook of research for educational communications and technology. (pp. 957-983). NewYork: Simon & Schuster Macmillan.Winn (2004). Cognitive perspectives in psychology. In D. H. Jonassen (Ed.). Handbook ofresearch for educational communications and technology (2 ed.). (pp. 79-112). New York:Simon & Schuster Macmillan.